Assessment of US Food and Drug Administration-Approved Digital Medical Devices for Just-in-Time Interventions: A Systematic Review.

Importance: Just-in-time interventions (JITIs) are a type of digital therapeutic intervention that combines remote monitoring tools and algorithms to personalize the delivery of specific interventions at the right time. The US Food and Drug Administration (FDA) regulatory approval documents are often the only available source of information on the effectiveness of therapeutic interventions based on these devices. Objective: To systematically review the publicly available information from the FDA on all recently approved medical devices used in JITIs to (1) assess how they operate to deliver JITIs and (2) appraise the evidence supporting their performance and clinical effectiveness. Evidence Review: Two reviewers systematically searched the Premarket Notifications (510(k)), Premarket Approvals, De Novo, and Humanitarian Device Exemption databases from January 2019 to December 2021 for all entries associated with devices that monitored patients' data over time to personalize the delivery of interventions to treat, prevent, or mitigate health conditions or events. They assessed whether the product summaries (1) enabled an understanding of how the device operated to deliver a JITI (eg, the nature, type, and frequency of the monitoring, the nature of the decision algorithm, and the nature and intended receiver of the intervention); (2) informed about the performance and effectiveness of the JITI; and (3) included information on data security and ownership. Findings: In total, 38 devices were included in this review. These were mainly intended for cardiac conditions (12 [31.6%]), diabetes (10 [26.3%]), and neurological diseases (4 [10.5%]). Monitoring devices ranged from wearable (18 of 28 [64.4%]; eg, smartwatches) to implanted sensors (6 of 28 [21.4%]; eg, inserted electrocardiographic sensors). Only 10 of 38 product summaries (26.3%) allowed a full understanding of how the device operated to deliver a JITI. Similarly, only 12 of 28 (42.9%), 12 of 36 (33.3%), and 5 of 38 (13.2%) reported the assessment of the performance of the monitoring device, assessment of the decision algorithm, and results of clinical studies assessing the effectiveness of the JITI, respectively. Finally, 14 of 36 product summaries (38.9%) included some information on data security, but none included information on data ownership. Conclusion and Relevance: The results of this systematic review suggest that the information publicly available in the FDA databases on the performance and effectiveness of digital medical devices used in JITIs is heterogeneous.

A systematic scoping review of just-in-time, adaptive interventions finds limited automation and incomplete reporting.

OBJECTIVES: To describe the degree of automation in just-in-time, adaptive interventions (JITAIs) assessed in randomized controlled trials (RCTs) in any medical specialty, and to assess the completeness of intervention reporting. STUDY DESIGN AND SETTING: Systematic scoping review-we searched PubMed, PsycINFO, and Web of Science, from 1 January 2019 to 2 March 2021, for reports of RCTs assessing JITAIs. We assessed whether study reports included the minimum information required to replicate the interventions based on JITAI frameworks. We described JITAIs according to their automation level using an established framework (partially, highly, or fully automated), and care workload distribution (requiring work from patients, health care professionals [HCPs], both, or neither). RESULTS: We included 88 JITAIs (62%, n = 55 supported chronic illness management and 12%, n = 11 supported health behavior change). Overall, 77% (n = 68) of JITAIs were missing some information required to replicate the intervention (e.g., n = 38, 43% inadequately reported the algorithm used to select intervention components). Only fifteen (17%) JITAIs were fully automated and did not require additional work from HCPs nor patients. Of the remaining JITAIs, 36% required work from both patients and HCPs, and 47% required work from either patients or HCPs. CONCLUSION: Most JITAIs are not fully automated and require work from the HCPs and patients.

Predictive model-based interventions to reduce outpatient no-shows: a rapid systematic review.

OBJECTIVE: Outpatient no-shows have important implications for costs and the quality of care. Predictive models of no-shows could be used to target intervention delivery to reduce no-shows. We reviewed the effectiveness of predictive model-based interventions on outpatient no-shows, intervention costs, acceptability, and equity. MATERIALS AND METHODS: Rapid systematic review of randomized controlled trials (RCTs) and non-RCTs. We searched Medline, Cochrane CENTRAL, Embase, IEEE Xplore, and Clinical Trial Registries on March 30, 2022 (updated on July 8, 2022). Two reviewers extracted outcome data and assessed the risk of bias using ROB 2, ROBINS-I, and confidence in the evidence using GRADE. We calculated risk ratios (RRs) for the relationship between the intervention and no-show rates (primary outcome), compared with usual appointment scheduling. Meta-analysis was not possible due to heterogeneity. RESULTS: We included 7 RCTs and 1 non-RCT, in dermatology (n = 2), outpatient primary care (n = 2), endoscopy, oncology, mental health, pneumology, and an magnetic resonance imaging clinic. There was high certainty evidence that predictive model-based text message reminders reduced no-shows (1 RCT, median RR 0.91, interquartile range [IQR] 0.90, 0.92). There was moderate certainty evidence that predictive model-based phone call reminders (3 RCTs, median RR 0.61, IQR 0.49, 0.68) and patient navigators reduced no-shows (1 RCT, RR 0.55, 95% confidence interval 0.46, 0.67). The effect of predictive model-based overbooking was uncertain. Limited information was reported on cost-effectiveness, acceptability, and equity. DISCUSSION AND CONCLUSIONS: Predictive modeling plus text message reminders, phone call reminders, and patient navigator calls are probably effective at reducing no-shows. Further research is needed on the comparative effectiveness of predictive model-based interventions addressed to patients at high risk of no-shows versus nontargeted interventions addressed to all patients.

Preferences for Alternative Care Modalities Among French Adults With Chronic Illness.

Importance: The COVID-19 pandemic led to the implementation of alternative care modalities (eg, teleconsultations and task shifting) that will continue to be implemented in parallel to traditional care after the pandemic. An ideal balance between alternative and traditional care modalities is unknown. Objectives: To quantify the ideal postpandemic balance between alternative and traditional care modalities among patients with chronic illness and to qualify the circumstances in which patients consider it appropriate to replace traditional care with alternative care. Design, Setting, and Participants: This survey study invited 5999 adults with chronic illness in ComPaRe, a French nationwide e-cohort of adults with chronic conditions who volunteer their time to participate in research projects, to participate in this study, which was performed from January 27 to February 23, 2021. Main Outcomes and Measures: Participants rated the ideal proportion at which they would use 3 alternative care modalities instead of the traditional care equivalent on a 0% to 100% scale (with 0% indicating using alternative care modalities for none of one's future care and 100% indicating using alternative care modalities for all of one's future care) of their overall future care: (1) teleconsultations, (2) online symptom-checkers to react to new symptoms, and (3) remote monitoring to adapt treatment outside consultations. The median ideal proportion of alternative care use was calculated. Perceived appropriate circumstances in which each alternative modality could replace traditional care were collected with open-ended questions. Analyses were performed on a weighted data set representative of patients with chronic illness in France. Results: Of the 5999 invited individuals, 1529 (mean [SD] age, 50.3 [14.7] years; 1072 [70.1%] female) agreed to participate (participation rate, 25.5%). Participants would choose teleconsultations for 50.0% of their future consultations (IQR, 11.0%-52.0%), online symptom-checkers over contacting their physician for 22.0% of new symptoms (IQR, 2.0%-50.0%), and remote monitoring instead of consultations for 52.3% of their treatment adaptations (IQR, 25.4%-85.4%). Participants reported 67 circumstances for which replacing traditional with alternative care modalities was considered appropriate, including 31 care activities (eg, prescription renewal and addressing acute or minor complaints), 25 patient characteristics (eg, stable chronic condition and established patient-physician relationship), and 11 required characteristics of the alternative care modalities (eg, quality assurance). Conclusions and Relevance: Results of this survey study suggest that after the pandemic, patients would choose alternative over traditional care for 22% to 52% of the time across different care needs. Participants proposed 67 criteria to guide clinicians in replacing traditional care with alternative care. These findings provide a guide for redesigning care in collaboration with patients after the pandemic.

Day-to-day discovery of preprint-publication links.

Preprints promote the open and fast communication of non-peer reviewed work. Once a preprint is published in a peer-reviewed venue, the preprint server updates its web page: a prominent hyperlink leading to the newly published work is added. Linking preprints to publications is of utmost importance as it provides readers with the latest version of a now certified work. Yet leading preprint servers fail to identify all existing preprint-publication links. This limitation calls for a more thorough approach to this critical information retrieval task: overlooking published evidence translates into partial and even inaccurate systematic reviews on health-related issues, for instance. We designed an algorithm leveraging the Crossref public and free source of bibliographic metadata to comb the literature for preprint-publication links. We tested it on a reference preprint set identified and curated for a living systematic review on interventions for preventing and treating COVID-19 performed by international collaboration: the COVID-NMA initiative (covid-nma.com). The reference set comprised 343 preprints, 121 of which appeared as a publication in a peer-reviewed journal. While the preprint servers identified 39.7% of the preprint-publication links, our linker identified 90.9% of the expected links with no clues taken from the preprint servers. The accuracy of the proposed linker is 91.5% on this reference set, with 90.9% sensitivity and 91.9% specificity. This is a 16.26% increase in accuracy compared to that of preprint servers. We release this software as supplementary material to foster its integration into preprint servers' workflows and enhance a daily preprint-publication chase that is useful to all readers, including systematic reviewers. This preprint-publication linker currently provides day-to-day updates to the biomedical experts of the COVID-NMA initiative.

Evaluation of Patient Willingness to Adopt Remote Digital Monitoring for Diabetes Management.

Importance: Patients will decide whether to adopt remote digital monitoring (RDM) for diabetes by weighing its health benefits against the inconvenience it may cause. Objective: To identify the minimum effectiveness patients report they require to adopt 36 different RDM scenarios. Design, Setting, and Participants: This survey study was conducted among adults with type 1 or type 2 diabetes living in 30 countries from February to July 2019. Exposures: Survey participants assessed 3 randomly selected scenarios from a total of 36. Scenarios described different combinations of digital monitoring tools (glucose, physical activity, food monitoring), duration and feedback loops (feedback in consultation vs real-time telefeedback by a health care professional or by artificial intelligence), and data handling modalities (by a public vs private company), reflecting different degrees of RDM intrusiveness in patients' personal lives. Main Outcomes and Measures: Participants assessed the minimum effectiveness for 2 diabetes-related outcomes (reducing hypoglycemic episodes and preventing ophthalmologic complications) for which they would adopt each RDM (from much less effective to much more effective than their current monitoring). Results: Of 1577 individuals who consented to participate, 1010 (64%; 572 [57%] women, median [interquartile range] age, 51 [37-63] years, 524 [52%] with type 1 diabetes) assessed at least 1 vignette. Overall, 2860 vignette assessments were collected. In 1025 vignette assessments (36%), participants would adopt RDM only if it was much more effective at reducing hypoglycemic episodes compared with their current monitoring; in 1835 assessments (65%), participants would adopt RDM if was just as or somewhat more effective. The main factors associated with required effectiveness were food monitoring (ß = 0.32; SE, 0.12; P = .009), real-time telefeedback by a health care professional (ß = 0.49; SE, 0.15; P = .001), and perceived intrusiveness (ß = 0.36; SE, 0.06; P < .001). Minimum required effectiveness varied among participants; 34 of 36 RDM scenarios (94%) were simultaneously required to be just as or less effective by at least 25% of participants and much more effective by at least 25% of participants. Results were similar for participant assessments of scenarios regarding the prevention of ophthalmologic complications. Conclusions and Relevance: The findings of this study suggest that patients require greater health benefits to adopt more intrusive RDM modalities, food monitoring, and real-time feedback by a health care professional. Patient monitoring devices should be designed to be minimally intrusive. The variability in patients' requirements points to a need for shared decision-making.

An International, Mixed-Methods Study of the Perceived Intrusiveness of Remote Digital Diabetes Monitoring.

OBJECTIVE: To assess the relationship between remote digital monitoring (RDM) modalities for diabetes and intrusiveness in patients' lives. PATIENTS AND METHODS: Online vignette-based survey (February 1 through July 1, 2019). Adults with diabetes (type 1, 2, or subtypes such as latent autoimmune diabetes of adulthood) assessed three randomly selected vignettes among 36 that combined different modalities for monitoring tools (three options: glucose- and physical activity [PA]-monitoring only, or glucose- and PA-monitoring with occasional or regular food monitoring), duration/feedback loops (six options: monitoring for a week before all vs before specific consultations with feedback given in consultation, vs monitoring permanently, with real-time feedback by one's physician vs by anoter caregiver, vs monitoring permanently, with real-time, artificial intelligence-generated treatment feedback vs treatment and lifestyle feedback), and data handling (two options: by the public vs private sector). We compared intrusiveness (assessed on a 5-point scale) across vignettes and used linear mixed models to identify intrusiveness determinants. We collected qualitative data to identify aspects that drove participants' perception of intrusiveness. RESULTS: Overall, 1010 participants from 30 countries provided 2860 vignette-assessments (52% were type 1 diabetes). The monitoring modalities associated with increased intrusiveness were food monitoring compared with glucose- and PA-monitoring alone (ß=0.34; 95% CI, 0.26 to 0.42; P<.001) and permanent monitoring with real-time physician-generated feedback compared with monitoring for a week with feedback in consultation (ß=0.25; 95% CI, 0.16 to 0.34, P<.001). Public-sector data handling was associated with decreased intrusiveness as compared with private-sector (ß=-0.15; 95% CI, -0.22 to -0.09; P<.001). Four drivers of intrusiveness emerged from the qualitative analysis: practical/psychosocial burden (eg, RDM attracting attention in public), control, data safety/misuse, and dehumanization of care. CONCLUSION: RDM is intrusive when it includes food monitoring, real-time human feedback, and private-sector data handling.

Changes in evidence for studies assessing interventions for COVID-19 reported in preprints: meta-research study.

BACKGROUND: The increasing use of preprints to disseminate evidence on the effect of interventions for the coronavirus disease 2019 (COVID-19) can lead to multiple evidence sources for a single study, which may differ in the reported evidence. We aim to describe the proportion of evidence on the effect of interventions for COVID-19 from preprints and journal articles and map changes in evidence between and within different sources reporting on the same study. METHODS: Meta-research study. We screened the Cochrane living systematic review and network meta-analysis (COVID-NMA) database to identify all preprints and journal articles on all studies assessing interventions for COVID-19 published up to 15 August 2020. We compared all evidence sources (i.e., preprint and associated journal article) and the first and latest versions of preprints for each study to identify changes in two evidence components: study results (e.g., numeric change in hazard ratio, odds ratio, event rate, or change in p value > or < 0.05 in any outcome) and abstract conclusions (classified as positive, negative or neutral regarding the intervention effect, and as reporting uncertainty in the findings or not). Changes in study results were further classified as important changes if they (1) represented a change in any effect estimate by ≥ 10% and/or (2) led to a change in the p value crossing the threshold of 0.05. RESULTS: We identified 556 studies. In total, 338 (61%) had been reported in a preprint: 66 (20%) of these had an associated journal article (median time to publication 76 days [interquartile range (IQR) 55-106]) and 91 (27%) had > 1 preprint version. A total of 139 studies (25% of the overall sample) were reported in multiple evidence sources or versions of the same source: for 63 (45%), there was a change in at least one evidence component between or within sources (42 [30%] had a change in study results, and in 29 [21%] the change was classified as important; 33 [24%] had a change in the abstract conclusion). For studies with both a preprint and an article, a median of 29% (IQR 14-50) of total citations were attributed to the preprint instead of the article. CONCLUSIONS: Results on the effect of interventions for COVID-19 are often reported in multiple evidence sources or source versions for a single study. Evidence is not stable between and within evidence sources. Real-time linkage of all sources per study could help to keep systematic reviews up-to-date.

A Methodologic Systematic Review of Mobile Health Behavior Change Randomized Trials.

CONTEXT: Mobile health helps providers offer accessible, affordable, tailored behavior change interventions. However, research assessing mobile health interventions may feature methodologic shortcomings and poor reporting. This review aims to summarize the characteristics, methods, and intervention reporting of RCTs evaluating mobile health behavior change interventions. EVIDENCE ACQUISITION: This was a methodologic systematic review of RCTs assessing mobile health behavior change interventions published in PubMed from January 1, 2014 to January 1, 2018, in journals with the upper half of Impact Factors (Clarivate Analytics). Three reviewers independently extracted sample characteristics. Primary outcomes were classified as patient-important or not using definitions from the literature. Any non-patient-important outcomes were then reclassified by a panel of 3 patients. Intervention reporting was assessed by the mobile health Evidence Reporting and Assessment checklist. Data were analyzed in December 2018. EVIDENCE SYNTHESIS: Most of the 231 included RCTs assessed text messaging (51%) or smartphone app (28%) interventions aiming to change nutrition and physical activity (36%) or treatment adherence (25%). Only 8% of RCTs had a patient-important primary outcome, follow-up of ≥6 months, and intent-to-treat analysis. Most primary outcomes were behavioral measures (60%). Follow-up was <3 months in 29% of RCTs. Regarding reporting, 12 of the 16 checklist items were reported in less than half of RCTs (e.g., usability/content testing, 32%; data security, 13%). CONCLUSIONS: Reports of RCTs assessing mobile health behavior change interventions lack information that would be useful for providers, including reporting of long-term intervention impact on patient-important primary outcomes and information needed for intervention replicability.

Reliability and Validity of the Greek Migraine Disability Assessment (MIDAS) Questionnaire.

BACKGROUND: The Migraine Disability Assessment (MIDAS) Questionnaire is a reliable and valid instrument for migraine-related disability. Such a tool is needed to quantify migraine-related disability in the Greek population. OBJECTIVE: This validation study aims to assess the test-retest reliability, internal consistency, item discriminant and convergent validity of the Greek translation of the MIDAS. METHODS: Adults diagnosed with migraine completed the MIDAS Questionnaire on two occasions 3 weeks apart to assess reliability, and completed the RAND-36 to assess validity. RESULTS: Participants (n = 152) had a median MIDAS score of 24 and mostly severe disability (58% were grade IV). The test-retest reliability analysis (N = 59) revealed excellent reliability for the total score. Internal consistency was α = 0.71 for initial and α = 0.82 for retest completion. For item discriminant validity, the correlations between each question and the total score were significant, with high correlations for questions 2-5 (range 0.67 ≤ r ≤ 0.79; p < 0.01). For convergent validity, there was significant negative correlation between the total score and all RAND-36 subscales except for 'emotional wellbeing'. The negative correlation indicates that patients with a lower degree of disability according to their MIDAS score tended to have better wellbeing. Psychometric properties are comparable with those of other published validation studies of the MIDAS and the original. Findings on question 1 show that missing work/school days may be closely related with increased affect issues. CONCLUSION: The Greek version of the MIDAS Questionnaire has good reliability and validity. This study allowed for cross-cultural comparability of research findings.